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Instructions for Completing Consolidated State Application 

Accountability Workbook 

By January 31 , 2003, States must complete and submit to the Department this 
Consolidated State Application Accountability Workbook. We understand that some of 
the critical elements for the key principles may still be under consideration and may not 
yet be final State policy by the January 31 due date. States that do not have final 
approval for some of these elements or that have not finalized a decision on these 
elements by January 31 should, when completing the Workbook, indicate the status of 
each element which is not yet official State policy and provide the anticipated date by 
which the proposed policy will become effective. In each of these cases, States must 
include a timeline of steps to complete to ensure that such elements are in place by 
May 1 , 2003, and implemented during the 2002-2003 school year. By no later than May 
1 , 2003, States must submit to the Department final information for all sections of the 
Consolidated State Application Accountability Workbook. 

Transmittal Instructions 

To expedite the receipt of this Consolidated State Application Accountability Workbook, 
please send your submission via the Internet as a .doc file, pdf file, rtf or .txt file or 
provide the URL for the site where your submission is posted on the Internet. Send 
electronic submissions to conapp@ed.gov. 

A State that submits only a paper submission should mail the submission by express 
courier to: 

Celia Sims 

U.S. Department of Education 
400 Maryland Ave., SW 
Room 3W300 

Washington, D.C. 20202-6400 
(202) 401-0113 
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PART I: Summary of Required Elements for State Accountability 
Systems 

Instructions 

The following chart is an overview of States' implementation of the critical elements 
required for approval of their State accountability systems. States must provide detailed 
implementation information for each of these elements in Part II of this Consolidated 
State Application Accountability Workbook. 

For each of the elements listed in the following chart, States should indicate the current 
implementation status in their State using the following legend: 

F: State has a final policy, approved by all the required entities in the State (e.g., 

State Board of Education, State Legislature), for implementing this element in its 
accountability system. 

P: State has a proposed policy for implementing this element in its accountability 

system, but must still receive approval by required entities in the State (e.g., 
State Board of Education, State Legislature). 

W: State is still working on formulating a policy to implement this element in its 

accountability system. 
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Summary of Implementation Status for Required Elements of 
State Accountability Systems 



Status 



Principle 1: All Schools 



State Accountability System Element 



F 1 .1 Accountability system includes all schools and districts in the state. 

F 1 .2 Accountability system holds all schools to the same criteria. 

F 1 .3 Accountability system incorporates the academic achievement standards. 

F 1 .4 Accountability system provides information in a timely manner. 

F 1.5 Accountability system includes report cards. 

F 1 .6 Accountability system includes rewards and sanctions. 



Pr nciple 2: All Students 

F 2.1 The accountability system includes all students 

F 2.2 The accountability system has a consistent definition of full academic year. 

F 2.3 The accountability system properly includes mobile students. 



Principle 3: Method of AYP Determinations 



F 3.1 Accountability system expects all student subgroups, public schools, and LEAs to reach 

proficiency by 2013-14. 

F 3.2 Accountability system has a method for determining whether student subgroups, public 
schools, and LEAs made adequate yearly progress. 

F 3.2a Accountability system establishes a starting point. 

F 3.2b Accountability system establishes statewide annual measurable objectives. 

F 3.2c Accountability system establishes intermediate goals. 

Pr nciple 4: Annual Decisions 

F 4.1 The accountability system determines annually the progress of schools and districts. 



STATUS Legend: 

F - Final state policy 

P - Proposed policy, awaiting State approval 
W - Working to formulate policy 
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Principle 5: Subgroup Accountability 

5.1 The accountability system includes all the required student subgroups. 

5.2 The accountability system holds schools and LEAs accountable for the progress of student 

subgroups. 

5.3 The accountability system includes students with disabilities. 

5.4 The accountability system includes limited English proficient students. 

5.5 The State has determined the minimum number of students sufficient to yield statistically 

reliable information for each purpose for which disaggregated data are used. 

5.6 The State has strategies to protect the privacy of individual students in reporting 
achievement results and in determining whether schools and LEAs are making adequate 
yearly progress on the basis of disaggregated subgroups. 

Pr nciple 6: Based on Academic Assessments 

F 6.1 Accountability system is based primarily on academic assessments. 

Principle 7: Additional Indicators 

F 7.1 Accountability system includes graduation rate for high schools. 

F 7.2 Accountability system includes an additional academic indicator for elementary and middle 

schools. 

F 7.3 Additional indicators are valid and reliable. 

Pr nciple 8: Separate Decisions for Readinq/Lanquaqe Arts and Mathematics 

F 8.1 Accountability system holds students, schools and districts separately accountable for 
reading/language arts and mathematics. 

Pr nciple 9: System Validity and Reliability 

F 9.1 Accountability system produces reliable decisions. 

F 9.2 Accountability system produces valid decisions. 

F 9.3 State has a plan for addressing changes in assessment and student population. 

Pr nciple 10: Participation Rate 

10.1 Accountability system has a means for calculating the rate of participation in the statewide 
assessment. 

1 0.2 Accountability system has a means for applying the 95% assessment criteria to student 
subgroups and small schools. 

STATUS Legend: 

F - Final policy 

P - Proposed Policy, awaiting State approval 
W- Working to formulate policy 



F 

F 
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PART II: State Response and Activities for Meeting State 
Accountability System Requirements 



Instructions 

In Part II of this Workbook, States are to provide detailed information for each of the 
critical elements required for State accountability systems. States should answer the 
questions asked about each of the critical elements in the State's accountability system. 
States that do not have final approval for any of these elements or that have not 
finalized a decision on these elements by January 31 , 2003, should, when completing 
this section of the Workbook, indicate the status of each element that is not yet official 
State policy and provide the anticipated date by which the proposed policy will become 
effective. In each of these cases, States must include a timeline of steps to complete to 
ensure that such elements are in place by May 1, 2003, and implemented during the 
2002-2003 school year. By no later than May 1 , 2003, States must submit to the 
Department final information for all sections of the Consolidated State Application 
Accountability Workbook. 
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PRINCIPLE 1. A single statewide Accountability System applied to all public 
schools and LEAs. 



CRITICAL ELEMENT 


EXAMPLES FOR 

MEETING STATUTORY REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1.1 How does the State 


Every public school and LEA is required to 


A public school or LEA is 


Accountability System 


make adequate yearly progress and is 


not required to make 


include every public 
school and LEA in the 


included in the State Accountability System. 


adequate yearly progress 
and is not included in the 


State? 


State has a definition of “public school” and 
“LEA” for AYP accountability purposes. 

• The State Accountability System 
produces AYP decisions for all public 
schools, including public schools with 
variant grade configurations (e.g., K- 
12), public schools that serve special 
populations (e.g., alternative public 
schools, juvenile institutions, state 
public schools for the blind) and public 
charter schools. It also holds 
accountable public schools with no 
grades assessed (e.g., K-2). 


State Accountability 
System. 

State policy systematically 
excludes certain public 
schools and/or LEAs. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



State law and regulations establish an accountability system that includes all public schools and districts 
in the state. 

Beginning in Cycle III (school years 2002/03 and 2003/04), K-2 schools with no MCAS results will be 
rated based on the grade 3 MCAS results of the schools their “graduates” attend. Other schools that lack 
reliable, comparable MCAS data as a result of small sample size (fewer than 20 students per year 
assessed in ELA or mathematics), or special circumstances (e.g., alternative schools that provide short- 
term educational services or provide services for excluded students, returning dropouts, or other special 
populations) may have AYP determinations based on data from sending or receiving schools to which 
enrolled students are linked or composite data reflecting district-wide results. MADOE shall determine, on 
a case by case basis, which score attribution method will result in AYP determinations that most 
accurately and reliably reflect the schools’ performance, and shall use that method. 



Evidence: Statute and regulations; Cycle II State Summary Report; Cycle II notice to K-2 schools. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1.2 How are all public schools 
and LEAs held to the same 
criteria when making an AYP 
determination? 


All public schools and LEAs are 
systematically judged on the 
basis of the same criteria when 
making an AYP determination. 

If applicable, the AYP definition is 
integrated into the State 
Accountability System. 


Some public schools and LEAs 
are systematically judged on the 
basis of alternate criteria when 
making an AYP determination. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


All schools and school districts are judged on the basis of the same criteria and receive annual AYP 
determinations based on those criteria. Student assessment results in ELA/Reading and mathematics on 
standard MCAS tests and MCAS-Alt Assessments are used to calculate a composite performance index 
for each school and district for students in the aggregate and for student subgroups. Beginning with Cycle 
III (years 2003 and 2004), attendance and graduation rate criteria have also been established and 
incorporated into the state’s AYP definition. The definition and determination of AYP is fully integrated 
into our state accountability system. 

Evidence: Statute and regulations; Cycle II explanatory materials (transmittal folder / power-point slides); 
Cycle III Mid-cycle explanatory materials. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .3 Does the State have, at a 
minimum, a definition of 
basic, proficient and 
advanced student 
achievement levels in 
reading/language arts and 
mathematics? 


State has defined three levels of 
student achievement: basic, 
proficient and advanced - 1 

Student achievement levels of 
proficient and advanced 
determine how well students are 
mastering the materials in the 
State’s academic content 
standards; and the basic level of 
achievement provides complete 
information about the progress of 
lower-achieving students toward 
mastering the proficient and 
advanced levels. 


Standards do not meet the 
legislated requirements. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


The Massachusetts School and District Accountability System holds schools and districts accountable for 
student performance on Massachusetts Comprehensive Assessment System (MCAS) results. MCAS is a 
custom designed assessment program based on MA student learning standards set out in State 
Curriculum Frameworks. MCAS assessments report student results in four performance categories: 
Advanced, Proficient, Needs Improvement, Failing (HSJ / Warning (elementary and middle grades). Our 
performance levels are similar to those used in reporting NAEP results. 


Evidence: MCAS explanatory materials posted on MADOE website. 





1 System of State achievement standards will be reviewed by the Standards and Assessments Peer 
Review. The Accountability Peer Review will determine that achievement levels are used in determining 
AYP. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .4 How does the State provide 
accountability and adequate 
yearly progress decisions 
and information in a timely 
manner? 


State provides decisions about 
adequate yearly progress in time 
for LEAs to implement the 
required provisions before the 
beginning of the next academic 
year. 

State allows enough time to 
notify parents about public school 
choice or supplemental 
educational service options, time 
for parents to make an informed 
decision, and time to implement 
public school choice and 
supplemental educational 
services. 


Timeline does not provide 
sufficient time for LEAs to fulfill 
their responsibilities before the 
beginning of the next academic 
year. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



MCAS tests are administered in the spring (April and May) to permit assessment of the full year of student 
attainment at the tested grade level. Tests include essay, open response and multiple-choice items. 

Open response items are scored over the summer, with teachers participating in the scoring process. 

The timeline for reporting MCAS results to schools, parents and the public has been accelerated over 
past two years by more than 60 days, from late November to mid-September. Beginning in 2004, the 
Massachusetts Department of Education will render preliminary AYP determinations and notify schools 
and districts of those determinations before the end of August of each year. We will require districts, 
upon receipt of this notice, to notify the parents of all students who are assigned to a school that has been 
preliminarily identified for improvement, corrective action or restructuring of their school choice option. 
Parent notification will, under this plan, take place no later than the first week of each school year, in time 
for alternative school assignments to be arranged if requested. 

Final school and district accountability reports and AYP determinations will be issued within 60 days after 
test results are returned. Once final accountability ratings and AYP determinations are issued we will 
revise the list of schools identified for improvement, corrective action or restructuring to reflect any 
additions or deletions resulting from analysis of the final determinations. Districts, upon receipt of final 
accountability results, will notify parents of the final results and make mid-year choice available in any 
cases where the preliminary AYP finding did not identify a school that, based on final results, is identified 
for improvement, corrective action or restructuring. In cases where a school was preliminarily identified 
but does not appear on the final list of schools identified for improvement, the school and its district will be 
so informed and the school will be relieved of prospective requirements. Any school choice commitments 
(i.e., transportation costs) that were made based on the preliminary identification will be honored for the 
balance of the school year. 

Evidence: MCAS release schedule 2002; SDAS Press release 2002 announcing accountability system 
results. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .5 Does the State 


The State Report Card includes 


The State Report Card does not 


Accountability System 


all the required data elements 


include all the required data 


produce an annual State 


[see Appendix A for the list of 


elements. 


Report Card? 


required data elements]. 


The State Report Card is not 




The State Report Card is 
available to the public at the 
beginning of the academic year. 

The State Report Card is 
accessible in languages of major 
populations in the State, to the 
extent possible. 

Assessment results and other 
academic indicators (including 
graduation rates) are reported by 
student subgroups 


available to the public. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



On an annual basis we publish a state profile, a state report card featuring detailed assessment data that 
meets all NCLB requirements, and a state AYP report. These documents are available on our website at 
http://profiles.doe.mass.edu/state.htm . 

We also publish, on an annual basis, school and district MCAS reports as well as individual student 
MCAS reports. Extensive information, including aggregate and disaggregated student assessment 
results and school and district performance ratings for every MA school and district is reported on DOE 
website as part of our state, district and school profiles. The SEA provides downloadable, print-ready 
LEA school report card data to LEAs on an annual basis for publication in locally produced district and 
school report cards. 

Beginning in 2003, student subgroup information, as well as aggregate information, is included in all 
school and district report cards, as well as in the MA state report card. Teacher data has been compiled 
locally for inclusion in 2003 and 2004 district and school report cards. MADOE is planning the 
development of a new teacher data warehouse. By 2006 we plan to have the data collection and storage 
systems in place to permit reliable State reporting of all teacher data we are required by federal law to 
include in annual state, district and school report cards. 



Evidence: Sample MCAS reports; Sample Web-version school and district profiles; mock-up of new 
report card format 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .6 How does the State 
Accountability System 
include rewards and 
sanctions for public schools 
and LEAs? 2 


State uses one or more types of 
rewards and sanctions, where 
the criteria are: 

• Set by the State; 

• Based on adequate yearly 
progress decisions; and, 

• Applied uniformly across 
public schools and LEAs. 


State does not implement 
rewards or sanctions for public 
schools and LEAs based on 
adequate yearly progress. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


State law provides for sanctions when schools or districts are determined to be under-performing. 
Sanctions range from required improvement planning with State oversight, to removal of school principal 
and reassignment of staff for schools and district receivership when under- performance is chronic. 
Recognition and rewards are also part of the MA school and district accountability system. Schools with 
exemplary improvement may be named Compass Schools. Compass schools receive $10,000 to assist 
with ongoing improvement initiatives and cover costs related to participation in effective practice 
dissemination activities. Privately funded cash awards are also given each year to principals of the most 
improved schools. 

Evidence: G.L. c. 69, sec. 1J an IK; 603 CMR 2.00 Regulations on Underperforming Schools and 
Districts; 2002 Pathways To Improved Student Performance report (listing 2001 and 2002 Compass 
Schools) 



2 The state must provide rewards and sanctions for all public schools and LEAs for making adequate 
yearly progress, except that the State is not required to hold schools and LEAs not receiving Title I funds 
to the requirements of section 1 1 1 6 of NCLB [§200.1 2(b)(40)]. 
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PRINCIPLE 2. All students are included in the State Accountability System. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


2.1 How does the State 


All students in the State are 


Public school students exist in 


Accountability System 


included in the State 


the State for whom the State 


include all students in the 


Accountability System. 


Accountability System makes no 


State? 


The definitions of “public school” 
and “LEA" account for all 
students enrolled in the public 
school district, regardless of 
program or type of public school. 


provision. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



All students enrolled in public schools and those being educated in private schools at public expense are 
required to participate in the MCAS. Most students participate by taking the standard form of MCAS 
tests. Make up sessions are given for students absent from school on scheduled testing dates. A variety 
of accommodations are available to permit most students with disabilities to participate in the standard 
form of the test. Students with severe and complex disabilities for whom participation in the standard 
form of the MCAS tests is not feasible or educationally appropriate participate in the MCAS Alternate 
Assessment program. 

Effective for 2004, we have adopted the new rule recently announced by USDOE for limited English 
proficient students who are enrolled in their first year in U.S. schools (“first-year LEP students”). First 
year LEP students are required to participate in state-mandated language proficiency assessments 
(MELA-O, LAS-R and LAS-W) and in MCAS mathematics tests. First year LEP students have the option 
but are not required to participate in standard ELA MCAS tests. MCAS mathematics and ELA test results 
(if any) for first-year LEP students will not be included when calculating a school, district, or state 
composite performance index for students in the aggregate or any student subgroup for Cycle III and 
subsequent review periods. 

In school year 2004/2005, we will begin administering the Massachusetts English Proficiency Assessment 
(MEPA), an alternative form of the MCAS ELA test specially designed to assess the level of ELA skills 
and knowledge of students with limited English proficiency. The MEPA test will assess performance on 
the same standards that are assessed through standard-form MCAS ELA tests. The MEPA, however, is 
designed to allow for more discriminating assessment of the development of English language skills with 
reference to the stages of knowledge and skill acquisition set forth in the Massachusetts English 
Language Proficiency Benchmarks and Outcomes for English Language Learners. Baseline MEPA 
performance data for individual students will be collected during a special MEPA administration in the fall 
of 2004. 



Evidence: MCAS and SDAS explanatory materials; Memo from Commissioner re: LEP testing and RFR 
for new ELP test design and administration 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


2.2 How does the State define 
“full academic year” for 
identifying students in AYP 
decisions? 


The State has a definition of “full 
academic year” for determining 
which students are to be included 
in decisions about AYP. 

The definition of full academic 
year is consistent and applied 
statewide. 


LEAs have varying definitions of 
“full academic year.” 

The State’s definition excludes 
students who must transfer from 
one district to another as they 
advance to the next grade. 

The definition of full academic 
year is not applied consistently. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


In October 2001 , the MADOE began collecting individual student data from local districts through our 
electronic Student Information Management System (“SIMS”). Districts are required to report student 
enrollment as of October 1 of each year. For purposes of the school and district accountability system, a 
student is considered to be enrolled in a school for a full year if he or she is enrolled as of October 1 of 
any school year and remains enrolled at that school up to and including the dates of MCAS test 
administration in the spring of that school year. 


Evidence: MCAS and SDAS explanatory materials 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


2.3 How does the State 
Accountability System 
determine which students 
have attended the same 
public school and/or LEA for 
a full academic year? 


State holds public schools 
accountable for students who 
were enrolled at the same public 
school for a full academic year. 

State holds LEAs accountable for 
students who transfer during the 
full academic year from one 
public school within the district to 
another public school within the 
district. 


State definition requires students 
to attend the same public school 
for more than a full academic 
year to be included in public 
school accountability. 

State definition requires students 
to attend school in the same 
district for more than a full 
academic year to be included in 
district accountability. 

State holds public schools 
accountable for students who 
have not attended the same 
public school for a full academic 
year. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


From 1 998 - 2002, test results for all students who took the MCAS tests at a particular school were 
included in determining the accountability system ratings and AYP determinations for that school. 

Effective for Cycle III (2003 and 2004 MCAS administrations), schools will only be accountable for the 
spring MCAS test results of students who, according to SIMS records, were enrolled in their school on 
October 1 of that school year. District accountability results will reflect performance of ALL students who 
took the MCAS tests while enrolled in the district’s schools, regardless of enrollment date in the district. 
Effective in 2004, MCAS ELA/reading and mathematics assessment results for limited English proficient 
student who are enrolled for the first year in U.S. schools will not be included in calculating the composite 
performance index values on which AYP determinations are based. 

Evidence: Law and regulations; Memo from Director of Data Collection, Analysis and Reporting 
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PRINCIPLE 3. State definition of AYP is based on expectations for growth in 
student achievement that is continuous and substantial, such that all students 
are proficient in reading/language arts and mathematics no later than 2013-2014. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.1 How does the State’s 

definition of adequate yearly 
progress require all students 
to be proficient in 
reading/language arts and 
mathematics by the 201 3- 
2014 academic year? 


The State has a timeline for 
ensuring that all students will 
meet or exceed the State’s 
proficient level of academic 
achievement in reading/language 
arts 3 and mathematics, not later 
than 2013-2014. 


State definition does not require 
all students to achieve 
proficiency by 2013-2014. 

State extends the timeline past 
the 201 3-2014 academic year. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



MA has established as a state goal having all students, in all schools, attain proficiency in ELA/reading 
and mathematics on or before the year 2014. Our accountability system is calibrated to measure annual 
performance toward the achievement of this objective. 

Effective in 2003, all MA public schools and school districts receive AYP determinations annually. The 
annual performance of each school and district for students in the aggregate and student subgroups that 
meet minimum size requirements is measured against MA performance targets derived from 
Massachusetts' NCLB “starting points” (additional detail in Elements 3.2(a) and 3.2(c) of this document). 
The sequence of biennial performance targets for the years 2002 - 2014 together are referred to as our 
“State Line.” The state line expresses the state’s performance expectations for each year, against which 
AYP determinations are made. Performance targets are raised in equal increments in each two- year 
cycle until the 2013-14 school year, by which time all students are expected to demonstrate proficiency in 
ELA/reading and mathematics. The performance of each school, district and the state is assessed by 
calculating a composite performance index (CPI). The CPI is a measure of the extent to which students 
have achieved or are progressing toward proficiency. Separate determinations are made for English 
language arts/reading and for mathematics. Assessment results for students in all grades within a school 
are combined to calculate the school's CPI for a given subject, group and time period. Beginning in 2006, 
we will issue district-level CPI calculations separately for the elementary, middle, and high school grade- 
spans (comprising grades 3-5, 6-8, and 9-12). 



3 If the state has separate assessments to cover its language arts standards (e.g., reading and writing), 
the State must create a method to include scores from all the relevant assessments. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2 How does the State 
Accountability System 
determine whether 
each student subgroup, 
public school and LEA 
makes AYP? 


For a public school and LEA to make 
adequate yearly progress, each student 
subgroup must meet or exceed the State 
annual measurable objectives, each student 
subgroup must have at least a 95% 
participation rate in the statewide 
assessments, and the school must meet the 
State’s requirement for other academic 
indicators. 

However, if in any particular year the student 
subgroup does not meet the annual 
measurable objectives, the public school or 
LEA may be considered to have made AYP, 
if the percentage of students in that group 
who did not meet or exceed the proficient 
level of academic achievement on the State 
assessments for that year decreased by 1 0% 
of that percentage from the preceding public 
school year; that group made progress on 
one or more of the State’s academic 
indicators; and that group had at least 95% 
participation rate on the statewide 
assessment. 


State uses different 
method for calculating how 
public schools and LEAs 
make AYP. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


Our AYP determination methodology was revised in 2002 to incorporate the new NCLB target timeline. 
Beginning with Cycle III (2003-2004) AYP is determined for students in the aggregate and for each 
student subgroup that meet minimum sample size requirements. AYP determinations are made 
separately for English language arts/reading and for mathematics, for students in the aggregate and for 
student subgroups that meet the minimum size requirements. 

We first take into consideration the student group’s rate of participation in MCAS for the year under 
review. Failure to meet the 95% participation requirement results in a negative AYP determination. 

If the 95% participation requirement is met, we next determine whether MCAS performance in English 
language arts/reading and in mathematics is at or above the state’s performance targets (intermediate 
goal) for those subjects for the year under review. 

When a group meets the 95% participation requirement and performs at or above the state target we next 
review the group’s performance on the applicable additional indicator. For all student groups within all 
schools and districts, performance and/or improvement on the applicable “additional indicator” 

(attendance for K-8 schools; graduation rate for high schools) is measured against the intermediate State 
targets that have been established for those indicators. A finding as to whether the applicable target was 
met by each student group is reported on school and district AYP reports. 
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A student group that met or exceeded 95% participation and performed at or above the state targets for a 
given year receives an affirmative AYP determination in that subject for that group as long as the group 
has met its target on the applicable additional indicator (K-8: attendance; 9-12: graduation rate). In 2004 
and future years MADOE will render a negative AYP determination when a student group fails to meet the 
applicable additional indicator target notwithstanding the fact that it meets or exceeds the applicable 
participation and performance targets. [Note: MADOE did not elect to make the required additional 
indicator an independent AYP variable for groups that meet participation and performance targets, but 
has been directed by USDOE to adopt this rule to comply with USDOE regulations.] 

When students in the aggregate or any student subgroup meet the assessment participation requirement 
in ELA or mathematics, but perform below the state target for that subject for the review period in 
question, we determine whether the NCLB safe-harbor provision applies. We measure the extent to 
which the school or district’s performance in that subject for that student group improved relative to its 
“baseline” performance in the previous two years. We determine whether the group, although performing 
below the applicable state target, has either a) decreased the percentage of students in that group who 
did not meet or exceed the proficient level of academic achievement on the State assessments for that 
year by 10% of that percentage from the preceding public school year; or b) has demonstrated a rate of 
improvement that is closing its performance gap at a rate that places it “on track” to achieving the goal of 
100% of students in that group attaining proficiency by the year 2014 (our USDOE-approved method for 
implementing the NCLB “safe-harbor” rule.) 

A student group that has performed below the state target but demonstrated sufficient improvement to 
qualify for “safe-harbor” must also meet the state’s performance or improvement target on the applicable 
additional indicator in order to receive an affirmative AYP determination. 

Beginning in 2006, we will make district-level AYP determinations separately in English language arts/ 
reading and mathematics for the elementary, middle, and high school grade-spans (comprising grades 3- 
5, 6-8, and 9-1 2). For a district to enter Improvement Status, the district must fail to make AYP in the 
same subject area (English language arts/reading or mathematics) in all three grade-spans for students in 
the aggregate or any subgroup for two consecutive years. 

School-level AYP determinations for English language arts/reading and math will continue to be based on 
all grades assessed in the school. 



Evidence: Cycle III informational handout and workshop presentation materials (PowerPoint slides). 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2a What is the State’s starting 
point for calculating 
Adequate Yearly 
Progress? 


Using data from the 2001-2002 
school year, the State 
established separate starting 
points in reading/language arts 
and mathematics for measuring 
the percentage of students 
meeting or exceeding the State’s 
proficient level of academic 
achievement. 

Each starting point is based, at a 
minimum, on the higher of the 
following percentages of students 
at the proficient level: (1 ) the 
percentage in the State of 
proficient students in the lowest- 
achieving student subgroup; or, 
(2) the percentage of proficient 
students in a public school at the 
20 th percentile of the State’s total 
enrollment among all schools 
ranked by the percentage of 
students at the proficient level. 

A State may use these 
procedures to establish separate 
starting points by grade span; 
however, the starting point must 
be the same for all like schools 
(e.g., one same starting point for 
all elementary schools, one same 
starting point for all middle 
schools...). 


The State Accountability System 
uses a different method for 
calculating the starting point (or 
baseline data). 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 
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EXAMPLES FOR 


EXAMPLES OF 


CRITICAL ELEMENT 


MEETING REQUIREMENTS 


NOT MEETING 






REQUIREMENTS 



We calculated our starting points for ELA and math using NCLB prescribed methodology (rank ordered 
lists for ELA and mathematics with determination of performance level of school representing 20 th 
percentile of state enrollment. 20 th percentile schools were higher performing in both ELA and 
mathematics than our lowest performing subgroup). 

Our State starting point for ELA was 39.7% proficient and advanced, which equates to a composite 
performance index of 70.7. For mathematics our starting point was 19.5% proficient and advanced, 
which equates to a composite performance index of 53. We used our State’s NCLB “starting points” as 
our Cycle II targets for determining AYP for the years 2001 and 2002. 

NOTE: For Cycle III, we revised our terminology to express our performance targets as “Composite 
Performance Index” or “CPI”, rather that “proficiency index” targets. This change in terminology was 
made to reflect our inclusion, beginning with Cycle III, of MCAS Alternate Assessment results in the 
indexing system on which our accountability determinations are based. 



Evidence: State Starting Point Determination slides; Cycle II explanatory materials. Cycle III 
informational handouts and Power Point slides. 



21 





CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2b What are the State’s annual 
measurable objectives for 
determining adequate yearly 
progress? 


State has annual measurable 
objectives that are consistent 
with a state’s intermediate goals 
and that identify for each year a 
minimum percentage of students 
who must meet or exceed the 
proficient level of academic 
achievement on the State’s 
academic assessments. 

The State’s annual measurable 
objectives ensure that all 
students meet or exceed the 
State’s proficient level of 
academic achievement within the 
timeline. 

The State’s annual measurable 
objectives are the same 
throughout the State for each 
public school, each LEA, and 
each subgroup of students. 


The State Accountability System 
uses another method for 
calculating annual measurable 
objectives. 

The State Accountability System 
does not include annual 
measurable objectives. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 
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EXAMPLES FOR 


EXAMPLES OF 


CRITICAL ELEMENT 


MEETING REQUIREMENTS 


NOT MEETING 






REQUIREMENTS 



We have established State targets for ELA and math performance for each two-year period from 2002 - 
2014. Our annual measurable objectives are for schools and districts to meet these targets. The targets 
progress in equal increments from our state starting points for ELA (CPI of 70.7) and Mathematics (CPI of 
53.0) to our goals, by 2014, of having all students performing at proficient and advanced levels in these 
subjects (CPI of 100 in both subjects). We have published and widely distributed our “State line,” which 
show the progression of specific performance targets for ELA/reading and mathematics that are in effect 
for each year from 2002 - 2014. 

To make AYP districts, schools, and subgroups must demonstrate student performance above the State 
target for that time period. If performance is below the state line a negative AYP determination is issued 
unless that school or district qualifies for “safe harbor” based on its rate of improvement (i.e., 
improvement at a rate that, projected forward, puts the school “on target” for getting all student to 
proficiency or above by 2014.) The minimum amount of improvement that a school and district is required 
to demonstrate to satisfy the safe harbor criteria can be calculated by dividing its performance gap for a 
particular subject and group (current CPI compared to CPI of 100), by the number of review cycles 
remaining until 201 4 when it is expected to have met the 1 00% proficiency goal. 

Specific annual performance objectives for Cycle III (2003 and 2004) for our additional indicators 
(attendance and graduation rate) were set by the MA Board of Education in the spring of 2002 after 
consultation and review by practitioners and the public. The Cycle III attendance target for elementary 
and middle schools is performance at or above 92% or improvement of 1 percentage point or greater 
compared to the prior year. The Cycle III grade 1 2 competency determination rate (the interim graduation 
rate measure adopted by the MA Board of Education) target for high schools is 70%. In 2005, the Board 
voted to maintain the same attendance and competency rate targets for Cycle IV (2005 and 2006). 



Evidence: State Starting Point Determination slides; Commissioner’s memo to Board of Ed 12/10/02. 
Memo from Commissioner Driscoll to MA Board of Education June 18, 2003 and minutes of June Board 
meeting documenting approval of proposed indicators and targets; Commissioner’s memo and minutes of 
June 2005 Board meeting. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2c What are the State’s 


State has established 


The State uses another method 


intermediate goals for 
determining adequate 


intermediate goals that increase 
in equal increments over the 


for calculating intermediate goals. 


yearly progress? 


period covered by the State 
timeline. 

• The first incremental 

increase takes effect not 
later than the 2004-2005 
academic year. 

• Each following incremental 

increase occurs within 
three years. 


The State does not include 
intermediate goals in its definition 
of adequate yearly progress. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 
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EXAMPLES FOR 


EXAMPLES OF 


CRITICAL ELEMENT 


MEETING REQUIREMENTS 


NOT MEETING 






REQUIREMENTS 



We have established intermediate State targets for MCAS ELA and math performance that apply to all 
schools and districts. We have set targets, expressed as composite performance index (“CPI”) values, for 
each two-year cycle from 2002 - 2014. A composite performance index of 100 indicates that 100% of 
students are performing at proficient or advanced levels. 

The MA “intermediate goals” for performance on MCAS assessments for 2004 - 2014 are: 

YEAR 





2004 


2006 


2008 


2010 


2012 


2014 


CPI 

ELA 


75.6 


80.5 


85.4 


90.2 


95.1 


100 


Mathematics 


60.8 


68.7 


76.5 


84.3 


92.2 


100 



We have also established intermediate performance targets for our additional indicators for Cycle III 
(years 2003 and 2004) and Cycle IV (years 2005 and 2006). 

The performance target for attendance for Cycle III and Cycle IV are average attendance of 92% or 1 
percentage point increase in the attendance rate compared to the prior year. We have established an 
interim graduation rate indicator for use until 2006-2007 when we will have the data needed to reliably 
calculate a four-year graduation rate. Our interim indicator is the grade 12 “competency determination” 
rate. Effective with the graduating class of 2003, MA students must earn a competency determination by 
passing state MCAS tests as a prerequisite for High School graduation. The state target competency 
determination or “CD” attainment rate for 2003 through 2006 is 70%. 

When a school or district fails to perform at or above the state target for the review period, the adequacy 
of a school’s improvement during the review period (safe harbor calculation) is assessed by comparing 
the rate of actual performance gain to the rate of improvement needed to meet the 2014 all students 
proficient goal 



Evidence: State Starting Point Determination slides; Commissioner’s memo to Board of Ed 12/10/02. 
Commissioner’s memo to Board and Board minutes, June 2003 and June 2005. 
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PRINCIPLE 4. State makes annual decisions about the achievement of all public 
schools and LEAs. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


4.1 How does the State 
Accountability System 
make an annual 
determination of whether 
each public school and LEA 
in the State made AYP? 


AYP decisions for each public 
school and LEA are made 
annually. 4 


AYP decisions for public schools 
and LEAs are not made annually. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



4 Decisions may be based upon several years of data and data may be averaged across grades within a 
public school [§1111 (b)(2)(J)]. 
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EXAMPLES FOR 


EXAMPLES OF 


CRITICAL ELEMENT 


MEETING REQUIREMENTS 


NOT MEETING 
REQUIREMENTS 



In 2003, we began publishing AYP determinations for all schools and districts on an annual basis. Those 
results are included in annual state, district and school level report cards. On an annual basis schools 
and districts that have failed to make AYP for two or more consecutive years are notified of the 
sanctions/requirements to which they are subject as a result of their identification for improvement, 
corrective action or restructuring. 

Continuing the school accountability practices and procedures MA established in 1999 and refined in 

2002 to meet NCLB requirements, in 2004 and future even-numbered years we intend to issue end-of- 
cycle performance and improvement ratings based on aggregate student performance, and annual AYP 
determinations for schools and districts. AYP determinations for ELA/reading and mathematics, which 
will reflect both aggregate and subgroup student performance, will be incorporated into these end-of-cycle 
reports. 

Again in 2005, and continuing in each odd numbered year thereafter, we will issue “mid-cycle” AYP 
determinations for all schools and districts. Determinations will be made, for students in the aggregate 
and student subgroups, in accordance with the procedures and reporting rules approved by USDOE in 

2003 for the first issuance of MADOE mid-cycle AYP determinations. For mid-cycle determinations we 
will review progress in the first year of the rating period toward achievement of the targets for that cycle. 
AYP determinations will be made by comparing mid-cycle performance to the State targets for each 
indicator. When performance at mid-cycle is found to be below the state target for that year, the rate of 
improvement will be examined to determine whether the safe-harbor rule applies. To make AYP at mid- 
cycle on the basis of improved student performance, a school or district must show improvement equal to 
or greater than 2/3 of their improvement target for the cycle. School and district improvement targets are 
a measure of the rate of improvement per year necessary to meet the goal of all students becoming 
proficient or advanced by 2014. 

Beginning in 2006, we will make district-level AYP determinations separately in English language arts/ 
reading and mathematics for the elementary, middle, and high school grade-spans (comprising grades 3- 
5, 6-8, and 9-12). School-level AYP determinations for English language arts/reading and math will 
continue to be based on all grades assessed in the school. 



Evidence: Planned SDAS Changes for Cycle III, Associate Commissioner Dow; Memo to 
Superintendents re: mid-cycle Supplemental Services requirements; Cycle III Power-point presentation 9- 
29-03. 
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PRINCIPLE 5. All public schools and LEAs are held accountable for the 
achievement of individual subgroups. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.1 How does the definition of 
adequate yearly progress 
include all the required 
student subgroups? 


Identifies subgroups for defining 
adequate yearly progress: 
economically disadvantaged, 
major racial and ethnic groups, 
students with disabilities, and 
students with limited English 
composite performance. 

Provides definition and data 
source of subgroups for adequate 
yearly progress. 


State does not disaggregate data 
by each required student 
subgroup. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


Since 2001 we have reported subgroup results on MCAS tests. Using our new Student Information 
Management System, we are able to match student data with test results and tabulate results for all 
required subgroups. Beginning in 2003, we issue subgroup AYP determinations for special education 
students, limited English proficient students, economically disadvantaged students, and students in racial 
and ethnic minority groups whenever there is a subgroup that meets our minimum sample size 
requirements. Subgroup AYP determinations are reflected in school and district and state accountability 
system reports beginning in 2003. 

Evidence: Sample MCAS reports; Memo from General Counsel confirming Board vote 12/18/02; 2003 
MCAS results posted on MADOE website at www.doe.mass.edu/mcas; Sample Cvcle III AYP 
Determination reports 



28 



CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.2 How are public schools 
and LEAs held 
accountable for the 
progress of student 
subgroups in the 
determination of adequate 
yearly progress? 


Public schools and LEAs are held 
accountable for student subgroup 
achievement: economically 
disadvantaged, major ethnic and 
racial groups, students with 
disabilities, and limited English 
proficient students. 


State does not include student 
subgroups in its State 
Accountability System. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Using our new Student Information Management System, we are able to match student data with test 
results and tabulate results for all required subgroups. For each school and district and for the state as a 
whole, subgroup AYP determinations are issued using the same decision and reporting rules that apply to 
AYP determinations for students in the aggregate. 

As described in Elements 3.1 , 3.2, and 4.1 , beginning in 2006 we will make district-level AYP 
determinations separately in English language arts/ reading and mathematics for the elementary, middle, 
and high school grade-spans (comprising grades 3-5, 6-8, and 9-12) for students in the aggregate and all 
subgroups. For a district to enter Improvement Status, the district must fail to make AYP in the same 
subject area (English language arts/reading or mathematics) in all three grade-spans for students in the 
aggregate or any subgroup for two consecutive years. 



Evidence: Sample MCAS reports; Memo from General Counsel confirming Board vote 12/18/02 



29 





CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.3 How are students with 
disabilities included in the 
State’s definition of 
adequate yearly progress? 


All students with disabilities 
participate in statewide 
assessments: general 
assessments with or without 
accommodations or an alternate 
assessment based on grade level 
standards for the grade in which 
students are enrolled. 

State demonstrates that students 
with disabilities are fully included 
in the State Accountability 
System. 


The State Accountability System 
or State policy excludes students 
with disabilities from participating 
in the statewide assessments. 

State cannot demonstrate that 
alternate assessments measure 
grade-level standards for the 
grade in which students are 
enrolled. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Most students with disabilities participate in MCAS testing, with accommodations if required. MCAS 
results for all disabled students who take standard MCAS tests are included in the calculation of a school 
or district composite performance index for students in the aggregate and each subgroup to which a 
student with disabilities belongs. Students with severe and complex disabilities, including students with 
significant intellectual/cognitive impairments for whom the standard MCAS test is not appropriate 
participate in the MCAS Alternate Assessment program (MCAS-Alt). Since 2003, we have included 
MCAS-Alt results in the accountability system as part of our composite performance index as described in 
Section A below. 

Interim 2% Flexibility Rule - 2005 and 2006: 

After generating preliminary AYP determinations, we will identify the schools and districts statewide that 
did not make AYP in English language arts, mathematics, or both subjects for the students with 
disabilities (SWD) subgroup only. We will perform a secondary analysis for those schools and districts, 
assigning 1 00 proficiency index points to those students with disabilities in their assessment group, if any, 
who are included in the 2% “proxy group.” We propose to use as our 2% “proxy group” of student MCAS 
results to be treated as if proficient, the results of students we deem to be “candidates for modified 
achievement standards” based on current year SIMS and performance data. We will use student disability 
type, level of need, and actual current year assessment performance data to identify these candidates 
from among students with disabilities who took the standard form of the MCAS test. We will re-run the 
AYP determinations for those schools/districts using the modified PI points assigned for “proxy group” 
students with disabilities. If, as a result of the secondary analysis, a school or district meets or surpasses 
its AYP performance targets, the school or district will be considered to have made AYP for the current 
school year. Details of our selection process are offered in the step-by-step explanation in Section B 
below. 
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Section A 



In Massachusetts, the MCAS-Alt assessment, a portfolio review process, is used to assess the attainment 
of students who, by reason of severe and complex disabilities, are not able to participate in the standard 
MCAS testing program. The MCAS-Alt is designed to both assess students who have significant 
cognitive impairments and are engaged with learning objectives distinct from those of age/grade level 
peers, and to measure the academic performance and progress of students whose disabilities do not impair 
cognitive functioning and are working toward attainment of grade level proficiency. The assessment 
results of MCAS-Alt participants who are not significantly cognitively impaired are included in the 
calculation of a school’s or district’s composite performance index (CPI) by measuring performance 
against the same standards and performance levels and assigning the same proficiency points per student 
that apply to students who take standard MCAS tests. 

During Cycle II (2001 and 2002) we introduced the “Proficiency Index” - a measure of the distribution of 
student performance in a school relative to attaining grade-level proficiency. A school’s proficiency 
index reflects student performance only on standard MCAS tests scored with reference to grade-level 
student learning standards. Beginning in 2003, we expanded the measures we use to assess school and 
district performance and improvement to include data from the MCAS-Alt assessment program, 
introducing the “composite performance index” as our new unit of school and district performance 
measurement. 

In 2003 and again in 2004, approximately one percent of students in Massachusetts public schools 
participated in the MCAS-Alt assessment program. Most of the students that currently participate in 
MCAS-Alt assessments do have significant cognitive impairments, as indicated by the students’ disability 
types and level of special education service needs (reported in our SIMS system), and the evidence from 
the level of complexity of the learning tasks with which they are able to engage (evident in their 
assessment portfolios). We anticipate that the number of students participating in the MCAS-Alt will 
increase in future years as training in the use of the MCAS-Alt assessment process expands and school 
staff become aware that their schools and districts now receive equivalent credit, in the rendering of AYP 
determinations, for the academic gains achieved by students with significant cognitive impairments 
engaged in individually appropriate “off-grade-level” learning objectives and students toward grade-level 
proficiency. SIMS data for 2003 and 2004 lead us to conclude that many more students qualify for use 
of the MCAS-Alt than arc currently participating in this assessment program. 

Beginning with 2003 AYP determinations, the assessment results of students participating in MCAS-Alt 
assessments have been combined with student assessment results from standard MCAS tests to generate a 
“Composite Performance Index” on which AYP determinations are based. Table 1 below shows the 
point values associated with the performance levels that are used to express the results obtained on 
standard MCAS tests and from MCAS-Alt participants who are working toward proficiency on grade- 
level learning standards. Table 2 below shows the point values that are associated with specified levels 
of performance on the MCAS-Alt assessment demonstrated by students working to achieve learning 
objectives, in accordance with their individual education plans, below grade-level for their age group. 

A school’s or district’s Composite Performance Index in each subject for each student group is calculated 
as follows: 

1) the number of students who took standard MCAS tests who performed at each of the five 
proficiency levels set out in Table 1 is multiplied by the number of proficiency index points 
associated with each level; 

2) the number of students who participated in the MCAS-Alt for reasons other than significant 
cognitive impairments who demonstrated performance equivalent to each of the five proficiency 
levels set out in Table 1 is multiplied by the proficiency index points associated with each level; 
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3) the number of students with significant cognitive impairments who demonstrated performance at each 
of the five MCAS-Alt levels shown in Table 2 is multiplied by the MCAS-Alt index points 
associated with each level; 

4) The point totals from steps one, two and three above are added together. The sum is divided by the 
total number of students assessed (standard MCAS tests and the MCAS-Alt.) 

The result is a number between 0 and 100 which constitutes the school’s or district’s Composite 

Performance Index (“CPI”) for that subject, group and time period. 

Table 1: Proficiency Index Points Table 2: MCAS-Alt Index Points 



For students taking standard MCAS tests and 
MCAS-Alt participants who do not have significant 
cognitive impairments 


MCAS SCALED SCORE or 
MCAS-Alt Equivalent 


POINTS 

AWARDED 


200 - 208 
Failing/Warning 


0 


210-218 

Failing/Warning - High 


25 


220 - 228 

Needs Improvement 


50 


230 - 238 

Needs Improvement - High 


75 


240 - 280 
Proficient/Advanced 


100 



For students with significant cognitive 
impairments taking MCAS-Alt 

(up to 1% of all assessed students) 


MCAS-ALT SCORE 


POINTS 

AWARDED 


Portfolio not submitted 


0 


Portfolio Incomplete 


25 


Awareness 


50 


Emerging 


75 


Progressing 


100 



Section B 



Interim 2% Flexibility Rule (2005 & 2006) - Step-Bx-Step Explanation 

1 . Assign Alternate Index (AI) points to all students whose performance was measured in relation to 
alternate achievement standards using the current year MCAS-Alt. 

2. Calculate 2.0 percent of total number of students assessed statewide in current year - in MCAS 
English language arts and mathematics. This figure represents the maximum number of students 
at the state level whose assessment results can be treated as if proficient under the flexibility 
available to students who would be appropriately assessed based on modified achievement 
standards. 

3. In the event that the number of students receiving AI points exceeds 1.0 percent of the total 
number of students assessed statewide in current year in MCAS English language arts and 
mathematics, subtract the excess number of student records from the 2.0 percent figure calculated 
in Step 2. Maintain AI points, as assigned in Step 1, for any excess number of student records. 

4. Use March SIMS disability incidence data and Spring standard MCAS results to identify up to 
the maximum number of students from Step 2, in ELA and math separately. Based on the criteria 
below, flag individual student records in the state student records database: 

a. All disabilities with high level of need scoring less than 220 points on current year 
MCAS 

b. All disabilities with moderate level of need scoring less than 210 points on current year 
MCAS 
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c. Specific disabilities with moderate level of need scoring between 210 and 218 points on 
current year MCAS 

i. Specific disabilities: Intellectual; Sensory/Deaf Blindness; Multiple Disabilities; 
Autism; Developmental Delay; Emotional Disability or Specific Learning 
Disability, if necessary. 

5. Once current year MCAS data is received from testing contractor, generate preliminary AYP 
calculations. 

6. Identify schools and districts that did not make AYP in current year for SWD subgroup only. 

7. Assign 100 PI points to targeted students from Step 4 who were enrolled and assessed in schools 
and districts that did not make AYP for the SWD subgroup only. 

8. Perform secondary analysis: re-generate current year CPI calculations using targeted students’ 
modified PI scores. Compare new CPI calculations to state performance targets without using 
confidence intervals or other statistical treatments. 

9. Determine an adjusted AYP determination for any SWD subgroup that meets or surpasses the 
state performance target based on the group’s modified CPI score in the secondary analysis. 

10. Generate AYP reports showing original (un-modified) CPI calculations, original AYP 
determinations, and adjusted AYP determinations, if any. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.4 How are students with 
limited English proficiency 
included in the State’s 
definition of adequate 
yearly progress? 


All LEP students participate in 
statewide assessments: general 
assessments with or without 
accommodations or a native 
language version of the general 
assessment based on grade level 
standards. 


LEP students are not fully 
included in the State 
Accountability System. 




State demonstrates that LEP 
students are fully included in the 
State Accountability System. 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



In 2003, all LEP students were required to participate in the MCAS assessment program, either by taking 
the standard form of the test or, in the case of LEP students with disabilities, the MCAS-Alt, with limited 
exceptions made for LEP students who entered US schools for the first time after October 1 , 2002 and 
lack a sufficient command of English to engage with the MCAS test material. 

For the 2004 MCAS administration, we have revised our test participation requirements in accordance 
with recently announced USDOE rule changes. For 2004 and subsequent years, LEP students in their 
first year in US schools are required to participate in our state language acquisition tests and in MCAS 
mathematics and science assessments, and are permitted but not required to participate in MCAS 
English language arts/reading tests. The performance of LEP students in their first year in US schools is 
not included in calculating a school or district’s CPI for ELA/reading or mathematics. All first year LEP 
students are, however, considered as participants for the purpose of determining whether a group met its 
95% participation target. 

During 2002-03 and 2003-04 LEP students who are not in their first year in US schools are participating in 
MCAS by taking standard form MCAS tests in English. Beginning in 2004-2005, LEP students will 
instead take the new MEPA test, a version of the ELA test specifically designed to measure language and 
content acquisition by students with limited English composite performance. Results from this test will be 
included in school and district accountability determinations beginning in Cycle IV (2004-05 and 2005-06). 



Evidence: MCAS explanatory materials; 1 1/1 9/02 Memo from Commissioner Driscoll to Board of 
Education re: plans for testing LEP students; 2003 MCAS participation guidelines posted on MADOE 
website; Cycle III informational materials. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.5 What is the State’s 

definition of the minimum 
number of students in a 
subgroup required for 
reporting purposes? For 
accountability purposes? 


State defines the number of 
students required in a subgroup 
for reporting and accountability 
purposes, and applies this 
definition consistently across the 
State. 5 

Definition of subgroup will result in 
data that are statistically reliable. 


State does not define the required 
number of students in a subgroup 
for reporting and accountability 
purposes. 

Definition is not applied 
consistently across the State. 

Definition does not result in data 
that are statistically reliable. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



5 The minimum number is not required to be the same for reporting and accountability. 
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To achieve reliability, consistency and validity of accountability decisions, our minimum sample size rules 
for issuing performance ratings and AYP determinations for students in the aggregate and for student 
subgroups are as follows: 

Beginning in 2004, school and district aggregate AYP determinations will be issued whenever the 
aggregate number of students included in the performance calculation (exclusive of any LEP students 
who were in their first year in US schools and in the case of schools, any students enrolled after October 
1 st of that school year) is forty (40) or greater in each year for which performance data is being analyzed. 

• Mid-cycle AYP determinations (odd numbered years) are based on a single year of data with a 
minimum sample size of 40; 

• End-of-cycle performance ratings and AYP determinations (even numbered years) are based on 
two years of data averaged together with a minimum sample size of 40 students assessed in 
each year of the two year rating cycle. 

When the number of students assessed per year is fewer than 40 MADOE will determine, on a case-by- 
case basis, the most reliable, consistent and valid method to render an AYP determination for the school 
or district. Where no more statistically reliable method exists, MADOE may render a school or district 
aggregate AYP determination on a sample size of not fewer than twenty (20) students. 

Beginning in 2004, school and district AYP determinations for student subgroups will be issued whenever 
(1 ) there are 40 or more subgroup members, AND (2) the number of subgroup members is at least five 
percent (5%) of students whose assessment results are included in the school’s or district’s aggregate 
AYP calculation OR (3) the number of subgroup members is 200 or more. 

• At mid-cycle, when AYP determinations are based on a single year of data, to receive an AYP 
determination a subgroup must meet the sample size requirements stated above for the year 
under review. 

• At end-of-cycle, when data for the two years comprising the review cycle are combined to 
determine performance ratings and AYP determinations, to receive a subgroup AYP 
determination a subgroup must meet the sample size requirements in each year of the two years 
under review. 

When determining the extent of improvement demonstrated (for purposes of applying the safe-harbor 
rule) we apply an error band of 2.5 points when school groups are composed of 100 or more students. 
This represents a typical 95% confidence interval for groups of that size. For groups with fewer than 1 00, 
but more than 20 students, we determine the actual 95% confidence interval for that group, up to a 
maximum of 4.5 CPI points. 

For districts with fewer than 100 students in an identified group, the error band rules used for school 
calculations apply. When district sample size is 1 00 or more but fewer than 400, a 2.5 point error band is 
applied when determining the adequacy of improvement for purposes of applying the safe-harbor rule. 
When sample size is 400 to 799, a 2.0 point error band is used. An error band of 1 .5 points are applied to 
district groups with 800 or more students; a 1 .0 point error band is used for groups larger than 1 ,200. 



Evidence: Cycle II and Cycle III explanatory materials. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.6 How does the State 
Accountability System 
protect the privacy of 
students when reporting 
results and when 
determining AYP? 


Definition does not reveal 
personally identifiable 
information. 6 


Definition reveals personally 
identifiable information. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


To ensure confidentiality of individual student information we do not report results for groups less than 10. 


Evidence: MCAS explanatory materials. 





6 The Family Education Rights and Privacy Act (FERPA) prohibits an LEA that receives Federal funds 
from releasing, without the prior written consent of a student’s parents, any personally identifiable 
information contained in a student’s education record. 
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PRINCIPLE 6. State definition of AYP is based primarily on the State’s academic 
assessments. 

EXAMPLES FOR 

CRITICAL ELEMENT MEETING REQUIREMENTS 



Formula for AYP shows that 
decisions are based primarily on 
assessments. 7 

Plan clearly identifies which 
assessments are included in 
accountability. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



For Cycles I and II, the MA school and district accountability system had student performance results on 
state MCAS tests as its only indicators. In conformance with NCLB, additional indicators have been 
added [graduation rate for High Schools and attendance at elementary and middle school levels] for 
Cycle III. 



Evidence: SDAS Framework and Cycle II explanatory materials; Cycle III informational handout and 
PowerPoint slides. 



7 State Assessment System will be reviewed by the Standards and Assessments Peer Review Team. 



6.1 How is the State’s 
definition of adequate 
yearly progress based 
primarily on academic 
assessments? 



EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 

Formula for AYP shows that 
decisions are based primarily on 
non-academic indicators or 
indicators other than the State 
assessments. 
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PRINCIPLE 7. State definition of AYP includes graduation rates for public High schools and an 
additional indicator selected by the State for public Middle and public Elementary schools (such 
as attendance rates). 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


7.1 What is the State definition 
for the public high school 
graduation rate? 


State definition of graduation rate: 

• Calculates the percentage 
of students, measured 
from the beginning of the 
school year, who graduate 
from public high school 
with a regular diploma (not 
including a GED or any 
other diploma not fully 
aligned with the state’s 
academic standards) in 
the standard number of 
years; or, 

• Uses another more 
accurate definition that 
has been approved by the 
Secretary; and 

• Must avoid counting a 
dropout as a transfer. 

Graduation rate is included (in the 
aggregate) for AYP, and 
disaggregated (as necessary) for 
use when applying the exception 
clause 8 to make AYP. 


State definition of public high 
school graduation rate does not 
meet these criteria. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



See USC 631 l(b)(2)(I)(i), and 34 C.F.R. 200.20(b) 
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EXAMPLES FOR 


EXAMPLES OF 


CRITICAL ELEMENT 


MEETING REQUIREMENTS 


NOT MEETING REQUIREMENTS 



At its December 2002 meeting the MA Board of Education approved modification of the MA SDAS to 
include graduation rates as an additional indicator for high schools. We will begin reporting graduation 
rates using the NCLB 4-year cumulative graduation rate definition in 2006-07, using SIMS data from 2002 
- 2006. The rate will be calculated as follows: 

# Graduates (with regular diploma) who completed high school in four years 

Divided by 

[# Graduates (same as above) + # of 9 th grade dropouts/retentions + # 10 th grade dropouts/retentions 

+ # 1 1 th grade dropouts/retentions + #12™ grade dropouts/retentions + # students who complete 1 2 th 

grade without a regular diploma] 

The data for each class will be tracked going forward from 9 th grade. Dropouts are defined as students 
who leave school prior to graduation for reasons other than transfer to another school. Students who are 
retained in grade, and thus leave their original class, will not count toward the number of graduates, but 
will be included in the denominator as members of the original class. 

In the interim, until SIMS data to accurately and reliably perform the calculation set out above becomes 
available, a school’s or district’s grade 12 competency determination or “CD” rate will be used as the 
graduation rate measure. [Note: We explored the feasibility of using SIMS data from 2001 to calculate a 
grade 10-12 graduation rate for use as our interim indicator, as projected in our initial NCLB 
Accountability workbook submission. We determined that the SIMS data reported during 2001 , the first 
year of implementation of our new state student management system) was not sufficiently complete and 
reliable to support valid accountability decision-making. For years previous to the 2002-2003 school year 
we were unable to consistently and reliably link data from our student data collection systems (SIMS) with 
state assessment (MCAS) data files, making it impossible to reliably track the progress of cohorts of high 
school students entering grade 9 before 2002. 

Moreover, the grade 12 competency determination rate is an appropriate interim graduation rate measure 
for our state because the class of 2003 was the first class required to meet a state “competency 
determination” standard (score of 220 or higher on state administered MCAS assessments in ELA and 
mathematics) as a requirement for graduation. We will continue to use the rate at which students attain a 
state competency determination as our additional accountability system indicator for high schools until 
2006-2007. 

The competency determination rate for a given year is calculated by determining how many students, 
exclusive of foreign exchange students, were enrolled in grade 1 2 as of October 1 of the school year 
(“a”), subtracting from this total those students who transferred into a different school between October 2 
and last day of that school year (“b”), determining how many from among the subset represented by “a” 
minus “b” successfully attained a competency determination by August 31 following their scheduled 
graduation date (“c”), then dividing “c” (numerator) by “a” - “b” (denominator). The rate is the percent of 
grade 12 students that met the MA graduation standard in the year under review. 



Evidence: Memo from General Counsel confirming Board vote; Memo from Director of Data Collection, 
Analysis and Reporting explaining graduation rate calculation method. Board memo and minutes, June 
2003; Cycle III informational handout and PowerPoint slides. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


7.2 What is the State’s 
additional academic 
indicator for public 
elementary schools for the 
definition of AYP? For 
public middle schools for 
the definition of AYP? 


State defines the additional 
academic indicators, e.g., 
additional State or locally 
administered assessments not 
included in the State assessment 
system, grade-to-grade retention 
rates or attendance rates. 9 


State has not defined an 
additional academic indicator for 
elementary and middle schools. 




An additional academic indicator 
is included (in the aggregate) for 
AYP, and disaggregated (as 
necessary) for use when applying 
the exception clause to make 
AYP. 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



At its December 2002 meeting the MA Board of Education approved modification of the MA SDAS to 
include attendance as an additional indicator for elementary and middle schools. After public consultation 
and Board discussion the decision was made to adopt average attendance, and improvement in average 
attendance as the additional indicator for elementary and middle schools. The performance target for this 
indicator is 92%; improvement target for those below 92% is 1 percentage point per year. Medically 
fragile students and students undergoing extended medical treatment as a result of serious illness or 
injury may, at the request of the LEA with supporting documentation, be excluded from the calculation of 
a school or district’s average attendance. 



Evidence: Memo from General Counsel confirming Board vote. June 2003 Board memo and minutes 



9 NCLB only lists these indicators as examples. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


7.3 Are the State’s academic 
indicators valid and 
reliable? 


State has defined academic 
indicators that are valid and 
reliable. 

State has defined academic 
indicators that are consistent with 
nationally recognized standards, if 
any. 


State has an academic indicator 
that is not valid and reliable. 

State has an academic indicator 
that is not consistent with 
nationally recognized standards. 

State has an academic indicator 
that is not consistent within grade 
levels. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


Beginning in 2001 , we now collect student data on an individual basis through the MA Student 
Information Management System (SIMS). The reliability of data reporting has been greatly enhanced 
through implementation of this system. 


Evidence: SIMS Data Standards Handbook (excerpt) 
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PRINCIPLE 8. AYP is based on reading/language arts and mathematics 
achievement objectives. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


8.1 Does the state measure 
achievement in 
reading/language arts and 
mathematics separately for 
determining AYP? 


State AYP determination for 
student subgroups, public 
schools and LEAs separately 
measures reading/language arts 
and mathematics. 10 

AYP is a separate calculation for 
reading/language arts and 
mathematics for each group, 
public school, and LEA. 


State AYP determination for 
student subgroups, public 
schools and LEAs averages or 
combines achievement across 
reading/language arts and 
mathematics. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


Yes. During Cycle 1 we reported results separately for ELA, math, and science, then combined data to 
create an “Overall” rating. For Cycle II, we dropped the overall ratings and only reported subject specific 
ratings and AYP determinations. In 2003 and going forward, we will issue separate ratings and AYP 
determinations for ELA and mathematics. 


Evidence: Cycle II explanatory material 





10 If the state has more than one assessment to cover its language arts standards, the State must create 
a method for including scores from all the relevant assessments. 
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PRINCIPLE 9. State Accountability System is statistically valid and reliable. 



CRITICAL ELEMENT 



9.1 HowdoAYP 

determinations meet the 
State’s standard for 
acceptable reliability? 



EXAMPLES FOR 
MEETING REQUIREMENTS 



State has defined a method for 
determining an acceptable level of 
reliability (decision consistency) 
for AYP decisions. 

State provides evidence that 
decision consistency is (1) within 
the range deemed acceptable to 
the State, and (2) meets 
professional standards and 
practice. 

State publicly reports the estimate 
of decision consistency, and 
incorporates it appropriately into 
accountability decisions. 

State updates analysis and 
reporting of decision consistency 
at appropriate intervals. 



EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 



State does not have an 
acceptable method for 
determining reliability (decision 
consistency) of accountability 
decisions, e.g.; it reports only 
reliability coefficients for its 
assessments. 

State has parameters for 
acceptable reliability; however, 
the actual reliability (decision 
consistency) falls outside those 
parameters. 

State’s evidence regarding 
accountability reliability (decision 
consistency) is not updated. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



We use two years of combined performance data to increase reliability during end of cycle 
determinations. The introduction in Cycle II of a proficiency index measurement tool, and further 
enhancement of that tool in Cycle III into what is now know as our “composite performance index” or 
“CPI” has enhanced the reliability of decisions compared to the scaled score criteria used in Cycle I. 

The CPI determinations, because they are based on increments that permit us to measure more discrete 
gradients of performance and track movement toward proficiency as well as attainment of proficiency, are 
also more stable than findings based simply on the percent proficient and advanced. 

The Department has invested considerable resources to develop our state’s student level data collection 
and management system (“SIMS”). This system has greatly enhanced our capacity to ensure that 
demographic data used to define subgroups is consistently reported and accurate. The enrollment 
information used in the accountability system is now directly connected to the MA school finance system, 
a change that has sharpened LEA attention to accuracy and timeliness in the reporting of student data. 



Evidence: Cycle II explanatory materials; SIMS information posted on DOE web-site. 
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EXAMPLES FOR EXAMPLES OF 

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING 

REQUIREMENTS 

9.2 What is the State's process State has established a process State does not have a system for 

for making valid AYP for public schools and LEAs to handling appeals of accountability 

determinations? appeal accountability decision. decisions. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Our assessment system results have been validated using comparisons with other standardized tests and 
review of alignment with State standards. These results are the primary indicators on which AYP 
determinations are made. We use minimum sample size rules to ensure the reliability of AYP findings. 

We look at both absolute performance and improvement to ensure that valid judgement are being 
rendered with respect to the adequacy of progress by a school toward achieving state and federal student 
performance goals. 



Evidence: MCAS technical documents and Achieve report published on DOE MCAS website 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


9.3 How has the State planned 
for incorporating into its 
definition of AYP 
anticipated changes in 
assessments? 


State has a plan to maintain 
continuity in AYP decisions 
necessary for validity through 
planned assessment changes, 
and other changes necessary to 
comply fully with NCLB. 11 

State has a plan for including new 
public schools in the State 
Accountability System. 

State has a plan for periodically 
reviewing its State Accountability 
System, so that unforeseen 
changes can be quickly 
addressed. 


State’s transition plan interrupts 
annual determination of AYP. 

State does not have a plan for 
handling changes: e.g., to its 
assessment system, or the 
addition of new public schools. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



11 Several events may occur which necessitate such a plan. For example, (1) the State may need to 
include additional assessments in grades 3-8 by 2005-2006; (2) the State may revise content and/or 
academic achievement standards; (3) the State may need to recalculate the starting point with the 
addition of new assessments; or (4) the State may need to incorporate the graduation rate or other 
indicators into its State Accountability System. These events may require new calculations of validity and 
reliability. 
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EXAMPLES FOR 


EXAMPLES OF 


CRITICAL ELEMENT 


MEETING REQUIREMENTS 


NOT MEETING 
REQUIREMENTS 



The composite performance index system allows us to incorporate results from new test administrations 
without disruption of accountability decisions. Our composite performance index allows us to combine 
results form standard MCAS tests with results from the MCAS-Alt measuring the educational progress of 
students with significant cognitive/intellectual impairments. We require two years of baseline data, and 
two years in the rating period as a basis for performance and improvement ratings and end-of-cycle AYP 
determinations. Mid-cycle AYP determinations are based on a single year’s data. 

In spring 2006 we implemented additional MCAS tests in English language arts and mathematics in 
grades 3 through 8 and 1 0 to meet the assessment requirements of NCLB. 



Grade 


English language arts 


Mathematics 


3 


Ongoing 


New in 2006 


4 


Ongoing 


Ongoing 


5 


New in 2006 


New in 2006 


6 


New in 2006 


Ongoing 


7 


Ongoing 


New in 2006 


8 


New in 2006 


Ongoing 


10 


Ongoing 


Ongoing 



Massachusetts averages two years of performance data when making AYP determinations in End-of- 
Cycle (even-numbered) years. As such, 2006 AYP CPI calculations will only include the results of those 
assessments that were also given during the 2004-05 school year. 

We will incorporate the results from all English language arts and mathematics assessments, in grades 3 
through 8 and 10, beginning with 2007 AYP calculations. 



Evidence: Cycle II explanatory materials; Cycle III explanatory materials; MCAS explanatory materials. 
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PRINCIPLE 10. In order for a public school or LEA to make AYP, the State 
ensures that it assessed at least 95% of the students enrolled in each subgroup. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 0.1 What is the State's method 
for calculating participation 
rates in the State 
assessments for use in 
AYP determinations? 


State has a procedure to 
determine the number of absent 
or untested students (by 
subgroup and aggregate). 

State has a procedure to 
determine the denominator (total 
enrollment) for the 95% 
calculation (by subgroup and 
aggregate). 


The state does not have a 
procedure for determining the 
rate of students participating in 
statewide assessments. 

Public schools and LEAs are not 
held accountable for testing at 
least 95% of their students. 




Public schools and LEAs are held 
accountable for reaching the 95% 
assessed goal. 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



We have reliable enrollment data from our student information system (SIMS). We cross match this data 
with MCAS testing program data. We routinely calculate and report participation rates in MCAS testing 
program. We began reporting participation rates on Cycle II accountability system reports. We have 
adjusted our methods for calculating participation to conform to NCLB rules. Students who do not 
participate in MCAS tests because of absence, whether excused or unexcused, are not considered to 
have participated, with the exception, beginning in 2004, of LEP students in their first year in US schools. 

Beginning in 2004, participation rates on the MCAS are being calculated as follows: 

# of students assessed + LEP students enrolled in US schools for the first time 

Divided by: 

# of students enrolled throughout the test administration window. 

Students who submit alternate assessment portfolios are included in the numerator and denominator. 
Students absent, but not medically excused, are in the denominator but not the numerator. 

In the event that a school or district, in a given year, fails to meet the 95% participation rate as a result of 
the unavailability of one or more students who were absent at the time of testing or unable to participate 
in the MCAS Alt Assessment by reason of a documented medical emergency occasioned by accident, 
illness or disability MADOE will review the matter on appeal and may recalculate the rate exclusive of 
those students medically unable to participate in the MCAS assessments. 



Evidence: MCAS reports; 2003 Mid-cycle AYP reports; accountability system technical documentation. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


10.2 What is the State’s policy 
for determining when the 
95% assessed 
requirement should be 
applied? 


State has a policy that 
implements the regulation 
regarding the use of 95% 
allowance when the group is 
statistically significant according 
to State rules. 


State does not have a procedure 
for making this determination. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


In Cycle II we began reporting participation rates on school rating reports. For Cycle III the 95% 
participation rate rule was applied in making all AYP determinations for students in the aggregate and 
student subgroups. 


Evidence: 2003 Mid-Cycle AYP Reports (posted on MADOE website). 
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Appendix A 

Required Data Elements for State Report Card 



1111(h)(1)(C) 

1 . Information, in the aggregate, on student achievement at each composite performance level on the 
State academic assessments (disaggregated by race, ethnicity, gender, disability status, migrant status, 
English composite performance, and status as economically disadvantaged, except that such 
disaggregation shall not be required in a case in which the number of students in a category is insufficient 
to yield statistically reliable information or the results would reveal personally identifiable information 
about an individual student. 

2. Information that provides a comparison between the actual achievement levels of each student 
subgroup and the State’s annual measurable objectives for each such group of students on each of the 
academic assessments. 

3. The percentage of students not tested (disaggregated by the student subgroups), except that such 
disaggregation shall not be required in a case in which the number of students in a category is insufficient 
to yield statistically reliable information or the results would reveal personally identifiable information 
about an individual student. 

4. The most recent 2-year trend in student achievement in each subject area, and for each grade level, 
for the required assessments. 

5. Aggregate information on any other indicators used by the State to determine the adequate yearly 
progress of students in achieving State academic achievement standards disaggregated by student 
subgroups. 

6. Graduation rates for secondary school students disaggregated by student subgroups. 

7. Information on the performance of local educational agencies in the State regarding making adequate 
yearly progress, including the number and names of each school identified for school improvement under 
section 1116. 

8. The professional qualifications of teachers in the State, the percentage of such teachers teaching with 
emergency or provisional credentials, and the percentage of classes in the State not taught by highly 
qualified teachers, in the aggregate and disaggregated by high-poverty compared to low-poverty schools 
which (for this purpose) means schools in the top quartile of poverty and the bottom quartile of poverty in 
the State. 



50 




